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FOREWORD 


Work  Unit  JACK  was  funded  under  Task  8d,  Modification  of  Training 
Courses,  Project  One  Hundred  Thousand.  The  funds  made  available  to 
HumRRO  under  task  8d  were  intended  primarily  to  provide  assistance  to 
the  U.S.  Army  Quartermaster  School  (USAQMS)  in  revision  of  the  Supplyman 
Course,  MOS  76A10.  However,  it  was  also  indicated  that  "As  an  adjunct 
to  this  program,  HumRRO  would  provide  technical  advisory  service  to  other 
schools  in  their  revision  of  similar  programs."  Under  this  provision, 
the  U.S.  Army  Southeastern  Signal  School  (USASESS)  requested  HumRRO  to 
provide  assistance  in  the  revision  of  the  Switchboard  Operators  Course, 

MOS  72C20.  In  April  1967,  agreement  was  reached  with  USASESS  that  HumRRO 
would  develop  a comprehensive  test  to  evaluate  the  effectiveness  of  a 
revised  course  for  Switchboard  Operators  prepared  by  USASESS. 

The  U.S.  Army  Research  Office  requested  that  the  efforts  for  QMS 
and  SESS  be  presented  in  the  HumRRO  FY  1968  Work  Program  as  Work  Units 
rather  than  Technical  Advisory  Service.  Accordingly,  the  QMS  effort  was 
designated  as  Work  Unit  STOCK;  the  SESS  effort  as  Work  Unit  JACK. 

The  products  of  Work  Unit  JACK  were  provided  to  SESS  on  a continuing 
basis  during  the  conduct  of  the  study  thus  precluding  the  need  for  a for- 
mal Consulting  Report.  The  nature  of  the  activities  does  not  warrant  prep- 
aration of  a Technical  Report.  Issuance  of  this  Staff  Paper  concludes 
Work  Unit  JACK. 


J.  DANIEL  LYONS 
Director  of  Research 


Division  No.  1 (System  Operations) 


The  Problem 


The  Department  of  Defense  and  particularly  the  Department  of  the 
Army,  in  the  belief  that  greater  utilization  could  be  made  of  lower 
aptitude  personnel  than  was  heretofore  the  case,  instituted  Project 
100,000.  CONARC  requested  HumRRO  Division  No.  1 to  provide  Technical 
Advisory  Service  in  support  of  two  efforts  relating  to  this  project. 

One  of  these  efforts,  eventually  known  as  Work  Unit  STOCK,  was  addressed 
to  the  Quartermaster  Supplyman  Course.  The  other  effort,  which  became 
Work  Unit  JACK,  was  addressed  to  providing  Technical  Advisory  Service  to 
the  United  States  Army  Southeastern  Signal  School  in  their  efforts  to 
adapt  the  Switchboard  Operators  Course,  MOS  72C20,  to  receive  Category 
IV  personnel  under  Project  100,000. 

In  mid-April  1967,  initial  negotiations  were  undertaken  between 
personnel  of  Division  1 and  USASESS  to  explore  the  level  of  effort  and 
the  role  proposed  for  Division  1 personnel  in  connection  with  the  school's 
objectives.  At  that  time  the  school  had  already  organized  a Project 
100,000  staff  headed  by  Mr.  Leon  Helmly  as  chief  of  a 5— man  task  force. 

One  of  the  objectives  of  the  USASESS  Project  100,000  effort  was  to 
revise  the  then  current  MOS  72C20  course  for  more  effective  training  of 
classes  containing  students  of  a broader  mental  mix.  This  broader  mental 
mix  would  result  from  acceptance  as  trainees,  personnel  whose  AFQT  scores 
fell  within  the  lower  end  of  Category  IV  and  who  previously  did  not 
qualify  for  military  service.  These  personnel  were  called  "new  accessions 
standards"  personnel  and  some  were  earmarked  (special  serial  number  assign- 
ment) as  Project  100,000  personnel  for  purposes  of  control  and  study.  The 
course  revision  was  a pilot  effort  to  identify  and  develop  the  method- 
ology and  techniques  for  revision  of  other  courses  also  selected  to  re- 
ceive classes  of  a broader  mental  mix.  The  course  revision  included  such 
actions  as  validation  of  course  objectives,  introduction  of  new  teaching 
aids  and  techniques,  and  evaluation  of  the  effectiveness  of  course 
improvements. 

The  specific  task  assigned  HumRRO  was  that  of  constructing  and 
administering  a performance  test  to  evaluate  the  effectiveness  of  the 
revised  72C20  course  in  meeting  its  stated  training  objectives. 

The  research  staff  was  eventually  responsible  for 

1.  Development  of  a performance  test  based  on  the  behavioral 
objectives  prepared  by  SESS  personnel  for  the  revised 
72C20  course 
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2.  Managing  the  conduct  of  the  test  Including 


r 


a.  Training  test  administrators 

b.  Scheduling  and  supervising  test  administration 

3.  Designing  and  conducting  the  statistical  analysis  of  the 
test  results 

4.  Preparing  a report  on  the  results  of  the  testing  program. 


The  Approach 

By  mid-May  the  research  staff  had  begun  to  collect  information 
relating  to  the  technical  aspects  of  the  hardware  the  switchboard  opera- 
tor would  be  trained  to  operate  and  maintain:  specifically,  the  tech- 
nical manuals  relating  to  the  switchboards,  the  generator,  and  certain 
references  relating  to  procedures.  The  determination  of  "what  to  test" 
and  performance  standards  were  based  on  the  detailed  course  terminal  and 
enabling  objectives  provided  to  the  research  staff  by  the  SESS  Project 
100,000  staff. 

The  course  objectives  forming  the  basis  of  the  test  design  were 
received  in  late  June.  By  13  September  tests  derived  from  the  duty- 
oriented  instructional  objectives  for  the  72C20  course  had  been  pre- 
pared. By  taking  advantage  of  the  redundancy  in  the  terminal  objectives 
for  the  separate  items  of  equipment  it  was  possible  to  construct  the 
test  that  could  be  conducted  in  a four  day  period,  although  the  length 
of  time  each  trainee  was  tested  was  81/2  hours.  The  equipments  uti- 
lized in  conducting  the  tests  were  SB-171,  SB-22/PT,  AN/TCC-7,  SB-86, 
AN/MTC-1,  AN/MTC-3,  PU-294,  and  TA-312. 

( 

During  the  period  of  14-17  August  selected  portions  of  the  test 
were  given  a dry  run  at  the  Southeastern  Signal  School.  Expert  opinion 
was  used  to  verify  test  procedures  and  time  limits  for  other  items.  In 
addition,  plans  were  made  to  pre-test  the  performance  test  during  the 
week  of  2-6  October.  The  first  two  pilot  study  classes  were  to  be 
tested  during  the  weeks  of  6-10  and  13-17  November.  School  personnel 
required  to  support  the  administration  of  the  test  were:  seven  instruc- 
tors, two  maintenance  men,  a supplyman,  a senior  NCO  to  serve  as  NCOIC, 
and  a vehicle  driver.  Care  was  taken  not  to  discuss  the  test  with  per- 
sonnel who  produced  behavioral  objectives  of  the  course  or  who  taught 

I the  course.  By  the  end  of  September  it  was  determined  that  the  try-out 

of  the  test  would  be  accomplished  during  the  first  shift  of  the  week 
2-6  October. 


Due  to  certain  unexpected  constraints  upon  equipment  and  test  sub- 
jects, the  scope  of  the  try-out  had  to  be  reduced.  However,  it  was 


possible  to  administer  each  test  item  at  least  once.  Test  item  defi- 
ciencies were  identified  and  corrected. 


Upon  the  instigation  and  recommendation  of  the  HumRRO  represen- 
tative, the  SESS  Director  of  Instruction  approved  a plan  to  gather 
performance  data  on  graduates  of  previous  72C20  courses.  The  HumRRO 
recommendation  was  based  upon  several  considerations:  (1)  the  objective 
of  the  course  revision  was  to  evaluate  the  effectiveness  of  course 
changes,  (2)  the  classes  presently  attending  the  standard  type  of  train- 
ing were  of  a broad  mental  mix  comparable  to  those  slated  for  entry  into 
the  revised  training,  and  (3)  the  class  (No.  34)  next  scheduled  for 
graduation  was  last  standard  class  which  could  be  considered  represen- 
tative of  "standard"  training.  The  last  three  scheduled  standard  classes 
(35,  36  and  37)  did  not  receive  typical  training  due  to  disruption  and 
equipment  shortages  caused  by  conversion  to  the  revised  training  course. 

Because  of  administrative  and  operational  constraints,  the  testing 
of  Class  No.  34  was  quite  limited  in  scope.  Only  the  three  tests  on 
switchboard  operation  were  administered  to  the  standard  course  graduates. 
These  tests  (#1,  #7,  and  #9)  contained  380  test  items.  The  factors  in- 
fluencing this  decision  were:  each  test  required  only  25  to  30  minutes 
to  administer,  adequate  equipment  and  space  was  available  and  did  not 
create  a conflict  with  the  on-going  training  program;  the  tests  sampled 
performance  of  tasks  considered  to  be  relatively  critical  to  job  success 
of  the  field.  The  three  performance  tests  administered  as  part  of  the 
standard  course  evaluation  program  were  very  similar  to  the  tests  selected 
and  sampled  the  same  tasks.  And  due  to  this  similarity  the  training  of 
the  test  administrators  on  these  items  could  be  accomplished  in  a rela- 
tively short  time. 

During  the  period  6-18  November  and  24  November  - 2 December,  40 
graduates  from  pilot  study  classes  38  and  40  were  administered  9 perfor- 
mance tests  and  1 written  test.  The  pilot  study  classes  were  specifically 
constituted  to  contain  a broad  mental  mix  of  students;  e.g.,  25%  were  to 
be  Project  100,000  personnel,  25%  were  AFQT  CAT  IV  but  not  Project  100,000 
personnel,  and  the  remaining  50%  above  CAT  IV  personnel.  Pilot  study 
class  38  was  administered  the  performance  test  during  the  week  of  13-17 
November.  Only  19  of  the  28  students  graduating  were  tested.  Among  the 
9 not  tested,  5 were  WACs,  three  were  discharged  prior  to  the  start  of 
testing  and  1 student  was  AWOL.  Graduates  of  class  39  were  not  tested 
because  of  the  limited  number  (13)  of  students  and  the  absence  of  any 
Category  IV  personnel.  During  the  period  of  27  November  to  1 December, 

21  graduates  of  class  40,  the  3rd  of  4 pilot  study  classes  were  tested. 
Four  graduates,  all  National  Guard  members  were  discharged  prior  to  the 
test. 

In  summary  then,  Division  1 research  personnel  were  asked  to  supply 
technical  advisory  service  to  the  Southeastern  Signal  School  in  preparing 
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and  administering  a performance  test  of  graduates  of  a revised  course  of 
instruction  in  switchboard  operation.  The  revised  course  was  designed 
to  provide  training  for  classes  containing  a broad  mix  of  personnel  by 
AFQT  level.  Portions  of  the  HumRRO  developed  test  were  also  administered 
to  graduates  of  a standard  course  to  compare  the  courses.  Administrative 
problems  prevented  testing  standard  training  personnel  on  the  entire  per- 
formance test. 

The  performance  test  consisted  of  10  tests  with  a total  of  550  items. 
It  was  administered  to  40  graduates.  Selected  parts  of  this  test  were 
also  administered  to  36  graduates  of  the  standard  72C20  course.  There 
was  a written  test  (15  items)  on  component  identification  and  circuit 
tracing.  The  other  9 tests  (535  items)  constituted  the  performance  test 
on  the  equipment.  Analysis  of  the  test  data  and  the  results  reported  to 
the  Southeastern  Signal  School. 

The  concluding  section  of  this  report  is  entitled  "Report  of  Treat- 
ment of  Data  from  the  Performance  Testing  of  Classes  38  and  40,  Revised 
Telephone  Switchboard  Operator  Course,  72C20."  This  section  presents 
the  summary  statistics  which  were  reported  to  the  Southeastern  Signal 
School  in  February  1968  in  the  format  largely  prescribed  by  them.  Since 
HumRRO  was  not  asked  for  comments  regarding  evaluation  or  interpretation 
of  the  data,  none  were  offered. 


Discussion 

Performance  testing  has  been  typically  time-consuming  and  expensive 
to  conduct.  The  history  of  the  development  and  the  conduct  of  the  test- 
ing under  JACK  is  another  case  supporting  that  view.  The  bringing  to- 
gether at  one  point  in  time  and  space  of  students,  test  administrators 
and  equipment,  while  at  the  same  time  deflecting  competing  demands  for  such 
personnel  and  materiel  resources  is  a major  managerial  activity.  Non- 
training aspects  of  the  soldiers  life  intrude  in  the  way  which  complicate 
the  task  of  scheduling  and  planning  for  training  operations.  That  is, 
medical  appointments,  guard  duty,  KP,  clean-up  details,  and  sundry  adminis- 
trative personnel  actions  take  precedence  over  the  trainee's  day  so  that 
the  ability  to  predict  numbers  of  personnel  available  for  certain  in- 
structional activities  is  well  nigh  impossible  much  less  prediction  of 
the  individual  by  name.  The  requirement  to  divert  equipment  normally 
used  for  training  to  testing  imposes  a severe  demand  upon  the  training 
system.  To  minimize  the  impact  of  the  loss  of  training  equipment  for 
classes  still  in  training  while  testing  was  taking  place,  rescheduling 
of  the  testing  to  non-training  periods  is  a possible  answer.  But  this 
in  itself  brings  to  play  other  aspects  which  complicate  the  lives  of  the 
test  administrators.  In  particular,  it  means  that  training  equipment 
has  to  be  altered  so  as  to  include  test  problems  prior  to  the  commencing 
of  testing  and  then  restored  to  operational  condition  prior  to  the 
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commencement  of  training  the  next  day.  But  problems  of  this  sort  are 
commonplace  and  form  the  typical  environment  in  which  work  of  this  sort 
is  carried  on. 


Copies  of  the  test  instruments,  schedule  and  test  results  have  been 
preserved  and  are  retained  in  the  division  records.  With  the  publica- 
tion of  this  Staff  Paper,  the  task  is  terminated. 

In  closing  it  is  interesting  to  note  that  in  the  Southeastern  Signal 
School's  report  to  CONARC  on  Project  100,000,  in  early  1968,  it  was 
contended  that  the  revised  course  was  effective  in  meeting  its  stated 
training  objectives,  that  groups  composed  of  a broad  mental  mix  were 
taught  effectively,  and  that  the  revision  of  the  training  resulted  in  an 
improvement  in  training  effectiveness  for  classes  of  a broad  mental  mix. 
Test  data  based  upon  the  81/2  hour  performance  test  developed  by  HumRRO 
under  JACK  do  not  support  those  contentions. 
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REPORT  OF  TREATMENT  OF  DATA  FROM  THE  PERFORMANCE 
TESTING  OF  CLASSES  #38  AND  #40,  REVISED 
TELEPHONE  SWITCHBOARD  OPERATOR  COURSE  (72C20) 


Enclosed  are  the  results  of  the  performance  testing  of  personnel 
trained  in  the  72C20  Standard  (Group  #34  Day  and  Night)  and  Revised 
(Group  #38  and  #40)  courses. 

There  was  a total  of  575  items  in  the  test.  The  items  were  grouped 
into  parts  and  the  parts  in  turn  into  tests;  1 written  (15  items  - 2 parts) 
and  9 performance  on  the  equipment  (70  parts  - 560  items).  Copies  of  the 
Administration  and  Score  Booklets  for  each  test  are  in  Appendix  A. 

The  tests  were  constructed  to  elicit  the  performance  behaviors 
described  in  Duty-Oriented  Behavioral  Objectives,  prepared  by  the 
developers  of  the  revised  72C20  course.  The  behaviors  in  a given  test 
or  part  of  a test  could  be  common  to  more  than  one  Behavioral  Objective; 
e.g.,  SB-86  switchboard  operation  was  in  two  Behavioral  Objective;  switch- 
board operation  of  (1)  the  SB-86,  and  (2)  the  MTC-3,  (same  switchboards  - 
different  housing).  Appendix  B shows  the  relationship  between  the  test 
items  and  the  Behavioral  Objectives. 

Each  test  or  test  part  had  a time  limit  placed  on  it.  The  time 
limit  for  the  test  "behaviors"  were  never  less  than  that  stated  in  the 
Behavioral  Objectives  of  the  revised  course.  The  man  being  tested  was 
checked  off  as  performing  each  item  correctly  or  incorrectly  (non- 
performance was  scored  "incorrect") . If  a man  failed  to  perform  all 
the  behaviors  in  a timed  part  within  the  allotted  time,  he  was  stopped 
and  received  a correct  score  only  for  the  behaviors  he  correctly  performed. 

Personnel  trained  in  the  revised  course,  hereafter  referred  to  as 
Group  #2  personnel,  were  tested  on  all  items,  parts  and  Behavioral 
Objectives.1  Personnel  trained  in  the  standard  course,  hereafter  referred 
to  as  Group  #1,  were  tested  on  some  parts  of  most  Behavioral  Objectives.2 

The  personnel  from  the  standard  course  who  were  tested  represented 
a sample  from  that  course.  There  is  no  reason  to  believe  that  the  sample 
was  not  representative  of  the  entire  group.  Those  not  tested  were  unavail- 
able because  of  administrative  reasons  relating  to  their  next  assignment 
and  not  related  to  their  class  standing  or  AFQT  scores. 


]_/  Behavioral  Objective  No.  7,  concerned  with  the  commercial  switch- 
board SB-171,  was  not  tested.  The  two  reasons  for  this  were  that  (1)  the 
SB-171  is  very  similar  to  the  SB-249  and  (2)  with  the  SB-171  equipment 
available  for  testing  purposes  it  would  not  have  been  administratively 
possible  to  test  all  behaviors. 

£/  Testing  of  the  standard  course  personnel  was  accomplished  during 
the  test  try  out.  The  availability  of  standard  trained  personnel  was  not 
sufficient  to  allow  testing  on  all  test  items. 


Personnel  have  been  grouped  into  three  categories  called  "III",  "IV" 
and  "100K."  Those  in  "III"  are  personnel  in  AFQT  categories  of  I,  II,  and 
III.  Those  in  "IV"  are  personnel  of  AFQT  category  IV.  Those  in  "100K" 
are  personnel  designated  as  project  one  hundred  thousand;  their  AFQT  scores 
are  predominantly  Category  IV. 

Table  1,  Appendix  C,  shows  the  mean  AFQT  scores  of  Groups  #1  and  #2. 

In  the  treatment  of  data,  Group  #1  (standard)  and  Group  # 2 (revised) 
personnel  are  compared  only  on  the  parts  of  the  test  which  both  groups 
took.  It  is  not  possible  to  state  how  well  the  standard  group  performed 
on  the  Behavioral  Objectives  because  none  of  the  students  were  tested  on 
all  parts. 

Tables  2,  3 and  4 in  Appendix  C contain  the  results  of  this 
comparison. 3 

There  were  no  personnel  'washed  out"  or  "washed  back"  in  the  revised 
course.  This  was  not  true  of  the  standard  course.  This  would  have  the 
effect  of  Inflating  the  average  performance  scores  of  the  personnel  trained 
in  the  standard  course  by  an  unknown  amount.  However,  the  three  lowest 
scores  for  Group  #2  on  all  four  tests  can  be  dropped  to  simulate  the  effect 
of  having  "washed  out"  the  "poorer"  students.  This  would  be  equivalent  to 
an  academic  attrition  rate  of  approximately  7%.**  Table  5,  Appendix  C, 
provides  the  results  of  the  comparison  when  attrition  was  taken  into 
account  for  Group  #2. 

Appendix  D contains  the  results  of  the  analyses  designed  to  provide 
answers  to  the  question  "Did  the  revised  course  meet  the  performance 
standards  for  its  Behavioral  Objectives?" 

Tables  1 and  2 report  the  results  of  the  F and  t tests  on  performance 
of  Group  #38  and  Group  #40  of  the  revised  course. 

Table  3 contains  a listing  of  the  cut-off  scores  for  each  sub- 
terminal objective  within  the  Duty-Oriented  Behavioral  Objectives. 

Table  4 provides  descriptive  statistics  for  evaluating  the  effect- 
iveness of  the  revised  course  in  meeting  its  stated  goals. 

In  Appendix  E are  the  results  of  the  correlations  between  performance 
tests  scores  and  selected  background  variables  of  personnel  from  the 
revised  course. 


V A composite  score  on  all  three  performance  tests  was  not  possible 
as  scores  from  test  #7  were  not  available  on  all  Group  #1  personnel. 

_/  The  average  academic  attrition  rate  for  the  72C20  course  (before 
revision)  was  reported  to  be  approximately  3 percent  and  average  class 
"wash  back"  between  6-7%. 
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APPENDIX  A 


TABLE  OF  CONTENTS 

TEST  # TITLE 

1 SB-249  Operation 

2 WTC-1 

3 SB-22  Installation  and  Maintenance 

4 SB-249  Maintenance 

5 AN/TTC-7  Component  Identification  and  Circuit  Tracing 

6 SB-86  Installation  and  Maintenance 

7 SB- 22  Operation  * 

8 MTC-3 

9 SB- 86  Operation 

10  PU-294  Operation  and  Maintenance 


APPENDIX  B 


I 


ASSIGNMENT  OF  TEST  ITEMS  TO  THE  BEHAVIORAL  OBJECTIVES 


TERMINAL  OBJECTIVE 


3-1-1  thru  3-1-3; 
3-2-1  thru  3-2-2 


3-3-1  thru  3-3-6; 
3-4-1  thru  3-4-6; 
3-8-1  thru  3-8-7 


3-5.1  thur  3-5.4 


5-6-1  thru  3-6-12; 
3-7-1  thru  3-7-8 


6-1-1  thru  6-1-4; 
6-1-7  8 6-1-8; 
6-3-4  thru  6-3-13 


6-1-5  8 6-1-6; 
6-6-1  thru  6-6-7; 
6-4-1  thru  6-4-6; 
6-3-1  thru  6-3-3 


6-5.1  thru  6-5.6 


6-3-14  thru  6-3-19 


5-A-l  thru  5-A-9 


5-B-l  thru  5-B-6 


TERMINAL  OBJECTIVE 

TEST 

3-c 

All  of  Test  #1 

3-d 

4-1  thru  4-9 

(All  of  Test) 

4-a 

8-1-1 

thru 

8-1-3; 

8-2-1 

thru 

8-2-7; 

8-3-1 

thru 

8-3-4; 

8-4-1 

thru 

8-4-2; 

8-5-1 

thru 

8-5-8 

— 

“ 4-b  

All  of  Test  #9 

4-c 

% 

6-5.1  thru  6-5.6 

4-d 

8-6-1  thru  8-6-4; 

8-7-1; 

6-3-4  thru  6-3-19; 

3-7-1  thru  3-7-8 

5-a 

2-1-1  thru  2-1-7;  2-2; 

2-3-1  5 2-3-2 

5-b 

All  of  Test  #1 

5-c 

2-4-1  thru  2-4-4; 

4-1  thru  4-9 

6-a 

All  of  Test  #1 

6-b 

4-1  thru  4-9 

8-a 

10-1-1  thru  10-1-4 

. 

8-b 

10-3-1 

thru 

10-3-10 

8-c 

10-2-1 

thru 

10-2-13 

\\ 
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APPENDIX  C 


TABLE  1 

MEAN  AFQT  SCORES 


GROUP 

APTITUDE  CATEGORIES 

# 

III 

IV 

100K 

TOTAL 

1 

49.46 

18.19 

15.14 

28.89 

2 

52.69 

21.00 

13.25 

28.97 

* Missing  AFQT  scores  were  reconstructed  by 
converting  selected  ACB  scores. 
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APPENDIX  C 


TABLE  2 - TREATMENT  OF  TEST  #5  DATA 

ANALYSIS  OF  VARIANCE 


A-  REVISED 

TRAINING 

CLASSES 

B-  APTITUDE  CATEGORIES 

III 

IV 

100K 

N=  4 

N=  9 

N=  6 

. CLASS  #38 

MEAN=  3.25 

MEAN=  2.89 

MEAN=  2.50 

S.D.=  1.89 

S.D.=  1.45 

S.D.=  0.55 

N=  10 

N=  5 

N=  6 

CLASS  #40 

MEAN=  3.20 

MEAN=  4.00 

MEAN-  1.17 

S.D.=  2.20 

S.D.=  3.46 

S.D.=  0.75 

P Probability 


A-  TRAINING  CLASSES 

0.887 

B-  APTITUDE  CATEGORIES 

0.094 

AB-  INTERACTION 

0.302 

BARLETT'S  TEST  = 17.408 

PROBABILITY  OF  CHI  SQUARE 

= 5 df,  0.0042 
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APPENDIX  C 


TABLE  3 - TREATMENT  OF  TEST  #7  DATA 


ANALYSIS  OF  VARIANCE 


A-  TYPE  OF 

TRAINING 

B-  APTITUDE  CATEGORIES 

III 

IV 

100K 

GROUP  #1 

N = 12 

MEAN  = 102.17 
S.D.  = 12.57 

N = 8 

MEAN  = 100.12 
S.D.  = 9.08 

N = 5 

MEAN  = 94.00 

S.D.  = 13.77 

GROUP  #2 

N = 14 

MEAN  = 99.36 
S.D.  = 9.37 

N = 14 

MEAN  = 95.79 
S.D.  = 9.23 

N = 12 

MEAN  = 86.25 

S.D.  = 7.03 

F Probability 


A-  TYPE  OF  TRAINING  0.066 


APTITUDE  CATEGORIES 

0.005 

INTERACTION 

0.740 

BARLETT'S  TEST  = 4.714 

PROBABILITY  OF  CHI  SQUARE  = 

5 df,  0.4519 

t TESTS 


COMPARISON  GROUPS 

t value 

df 

Level  of 
Significance 

All  of  Group  #1,  vs  all  of  Group  #2 

2.010 

63 

0.05 

100K,  Group  #1  vs.  100K,  Group  #2 

1.196 

15 

NS 

III  5 IV.  Group  #1  vs.  Ill  5 IV, Group  #2 

1.243 

46 

NS 

IV,  Group  #1  vs.  IV,  Group  #2 

1.072 

20 

NS 

III,  Group  #1  vs.  Ill,  Group  #2 

0.637 

24 

NS 

IV,  Group  #1  vs.  100K,  Group  #1 

0.882 

11 

NS 

III,  Group  #1  vs.  100K,  Group  #1 

1.143 

15 

NS 

III,  Group  #1  vs.  IV,  Group  #1 

0.421 

18 

NS 

IV,  Group  #2  vs.  100K,  Group  # 2 

2.984 

24 

0.01 

III,  Group  *2  vs.  100K,  Group  # 2 

4.066 

24 

0.001 

111,  Group  *2  vs.  IV,  Group  # 2 

1.016 

26 

NS 

1 A 


APPENDIX  C 


TABLE  4 - TPJEATMENT  OF  TEST  #1  5 #9  DATA 
ANALYSIS  OF  VARIANCE 


A-  TYPE  OF 

TRAINING 

B-  APTITUDE  CATEGORIES 

III 

IV 

100K 

GROUP  #1 

N = 14 

MEAN  = 213.93 
S.D.  = 14.05 

N = IS 

MEAN  = 201.93 
S.D.  = 17.40 

N = 7 

MEAN  = 202.00 
S.D.  = 24.70 

GROUP  #2 

N = 14 

MEAN  = 202.50 
S.D.  = 15.86 

N = 14 

MEAN  = 200.07 
S.D.  = 9.18 

N = 12 

MEAN  = 187.00 
S.D.  = 16.26 

F PrQbability 


A-  TYPE  OF  TRAINING  0.015 


APTITUDE  CATEGORIES 

0.016 

INTERACTION 

0.346 

BARLETT'S  TEST  = 9.104 

PROBABILITY  OF  CHI  SQUARE  = 5 df,  0.1051 


t TESTS 


COMPARISON  GROUPS 

t value 

df 

Level  of 
Significance 

All  of  Group  #1,  vs.  all  of  Group  #2 

2.476 

74 

0.02 

100K,  Group  #1  vs.  100K,  Group  #2 

1.435 

17 

NS 

III  5 IV,  Group  #1  vs.  Ill  § IV,  Group  #2 

1.636 

55 

NS 

IV,  Group  #1  vs.  IV,  Group  #2 

0.364 

27 

NS 

III,  Group  #1  vs.  Ill,  Group  #2 

2.018 

26 

NS 

IV,  Group  #1  vs.  100K,  Group  #1 

-0.006 

20 

NS 

III,  Group  #1  vs.  100K,  Group  #1 

1.185 

19 

NS 

III,  Group  #1  vs.  IV,  Group  #1 

i 

2.049 

27 

NS 

IV,  Group  #2  vs.  100K,  Group  #2 

2.468 

24 

0.05 

III,  Group  #2  vs.  100K,  Group  #2 

2.451 

24 

0.05 

III,  Group  #2  vs.  IV,  Group  #2 

0.496 

26 

NS 
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APPENDIX  C 


TABLE  5 - COMPARISON  OF  GROUP  #1  WITH  GROUP  #2  PERSONNEL  AFTER  AN 
ADJUSTMENT  FOR  ACADEMIC  ATTRITION  IN  GROUP  #2  HAS  BEEN 
MADE 


t TEST 


PERFORMANCE  MEASURES 

t value 

df 

Level  of 
Significance 

Test  #1  and  #9  combined 

1.929 

71 

NS 

Test  #5 

4.725 

49 

.001 

Test  #7 

1.729 

60 

NS 

A-  TYPE  OF  TRAINING 0. 

B-  APTITUDE  CATEGORIES 
AB-  INTERACTION 

BARLETT'S  TEST  = 8.691 
PROBABILITY  OF  OH  SQUARE  = 5 df,  0.1224 


t TESTS 


COMPARISON  GROUPS  t value 


All  of  Group  #1,  vs.  all  of  Group  #2  4.508 


100K,  Group  #1  vs.  100K,  Group  #2  2.381 


III  6 IV,  Group  #1  vs.  Ill  6 IV,  Group  #2  5.188 


IV,  Group  #1  vs.  IV,  Group  #2  3.747 


III,  Group  #1  vs.  Ill,  Group  #2  3.127 


IV,  Group  #1  vs.  100K,  Group  #1  3.479 


III,  Group  #1  vs.  100K,  Group  #1  2.782 


III,  Group  #1  vs.  IV,  Group  #1  -0.122 


IV,  Group  #2  vs.  100K,  Group  #2  2.161 


III,  Group  #2  vs.  100K,  Group  #2  2.264 


III,  Group  #2  vs.  IV,  Group  #2  -0.087 


0.001 


0.05 


0.001 


0.01 


0.01 


0.02 


0.05 


NS 


0.05 


0.05 


NS 
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APPENDIX  D 

TABLE  2 - TREATMENT  OF  TESTS  #1-10,  EXCLUDING  #5  DATA 


ANALYSIS  OF  VARIANCE 


REVISED 

TRAINING 

CLASSES 

APTITUDE  CATEGORIES 

III 

IV 

100K 

N = 4 

N = 9 

N = 6 

CLASS  #38 

MEAN  = 425.00 

MEAN  = 408.44 

MEAN  = 382.17 

S.D.  = 28.69 

S.D.  = 21.29 

S.D.  = 36.52 

N = 10 

N ■=  5 

N = 6 

CLASS  #40 

MEAN  = 417.70 

MEAN  = 406.80 

MEAN  <=  364.17 

S.D.  = 35.47 

S.D.  = 42.17 

S.D.  = 17.22 

F Probability 


TRAINING  CLASSES 0.396 

APTITUDE  CATEGORIES  0.002 

INTERACTION 0.809 

BARLETT'S  TEST  = 5.632 

PROBABILITY  OF  CHI  SQUARE  = 5 df,  0.3441 


COMPARISON  GROUPS 


All  of  Group  #1,  vs.  all  of  Group  #2 


100K,  Group  #1  vs.  100K,  Group  #2 


III  8 IV,  Group  #1  vs.  Ill  8 IV,  Group  #2 


IV,  Group  #1  vs.  IV,  Group  #2 


III,  Group  #1  vs.  Ill,  Group  #2 


IV,  Group  #1  vs.  100K,  Group  #1 


III,  Group  #1  vs.  100K,  Group  #1 


III,  Group  #1  vs.  IV,  Group  #1 


IV  vs.  100K 


III  vs.  100K 


III  vs.  IV 


t value 
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TABLE  3 - CUT-OFF  SCORES  FOR  TERMINAL  OBJECTIVES 


OBJECTIVE 

NUMBER  OF 

TEST  ITEMS 

ACCURACY 

STANDARD 

CUT-OFF 
SCORE  *’ 

#1 

a 

5 

80% 

4 

b 

19 

80%* 

15 

c 

129 

90% 

116 

d 

4 

80%  * 

3 

e 

20 

80%* 

16 

#2 

a 

16 

80% 

13 

b 

18 

* 

o 

oo 

14 

c 

126  i 

90% 

113 

d 

6 

80%* 

5 

e 

6 

* 

o 

00 

5 

#3 

a 

9 

7 of  9 

7 

b 

6 

4 of  6 

4 

c 

125 

90% 

112 

d 

9 

OO 

o 

* 

7 

#4 

a 

24 

90% 

22 

b 

126 

90% 

113 

c 

6 

80%* 

5 

d 

29 

* 

o\° 

O 

OO 

23 

#5 

a 

10 

90% 

9 

b 

• 125 

90% 

112 

c 

13 

80%* 

10 

#6 

a 

125 

90% 

112 

b 

9 

80%* 

7 

#8 

a 

4 

90% 

3*** 

b 

10 

90% 

9 

c 

13 

80%* 

10 

NOTES 

* Accuracy  standards  were  not  specified  in  the  "Duty-Oriented  Instruc- 
tional Objectives"  and  were  arbitrarily  set  at  80%  by  the  Research 
staff. 

**  Cut-off  scores  were  rounded  off  to  nearost  whole  number. 

***  Actually  represents  accuracy  standard  of  75%. 
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PERCENT  OF  REVISED  COURSE  PERSONNEL  MEETING  BEHAVIORAL  OBJECTIVE 

PERFORMANCE  STANDARDS 


TERMINAL 

OBJECTIVE 

CUT-OFF 

SCORE 

! 

PERCENT  MEETING  CUT-OFF  SCORE 

MEAN  SCORE  OF 

ALL  CATEGORIES 

H9 

IV 

N»14 

».  »U0J| 

mourn 

ALL 

N=40 

la 

4 

21.43 

r - - - 

r 21.43 

' 16.67 

20.00 

2.92 

lb 

15 

50.00 

35.71 

16.67 

- 

35.00 

13.17 

lc 

116 

7.14 

7.14 

0.0 

5.00 

94.17 

Id 

3 

7.14 

21.43 

8.33 

12.50 

0.95 

H- • 

G>* 

16 

7.14 

7.14 

0.0 

5.00 

11.30 

TOTAL 

OBJECTIVE 

154 

0.0 

7.14 

o 

o 

2.50 

122.52 

2a 

13 

35.71 

28.57 

8.33 

25.00 

10.05 

2b 

14 

50.00 

35.71 

16.67 

35.00 

12.32 

2c 

113 

21.43 

0.0 

0.0 

7.50 

102.63 

2d 

5 

14.29 

0.0 

■m 

5.00 

2.30 

2e 

5 

0.0  • 

0.0 

mu 

0.0 

1.65 

TOTAL 

OBJECTIVE 

150 

7.14 

0.0 

0.0 

2.50 

128.95 

3a 

7 

0.0 

0.0 

o 

O 

0.0 

2.70 

3b 

4 

0.0 

7.14 

0.0 

2.50 

0.63 

3c 

112 

7.14 

7.14 

0.0 

5.00 

94.37 

3d 

7 

7.14 

35.71 

16.67 

20.00 

4.65 

TOTAL 

OBJECTIVE 

130 

0.0 

0.0 

0.0 

0.0 

102.35 

4a 

22 

7.14 

0.0 

0.0 

2.50 

15.77 

I 

: 


l 

\ 


i 


TERMINAL 

OBJECTIVE 

CUT-OFF 

SCORE 

PERCENT  MEETING  CUT-OFF  SCORE 

MEAN  SCORE  OF 

ALL  CATEGORIES 

III 

IV 

100K 

ALL 

5a 

9 

0.0 

0.0 

.• _ . / ■■ 

0.0 

3.95 

5b 

112 

7.14 

7.14 

0.0 

5.00 

94.37 

5c 

10 

7.14  I 

35.71 

8.33 

17.50 

7.42 

TOTAL 

OBJECTIVE 

131 

0.0 

0.0 

0.0 

0.0 

105.75 

6a 

112 

7.14 

7.14 

0.0 

5.00 

94.37 

6b 

7 

7.14 

35.71 

16.67 

20.00 

4.65 

TOTAL 

OBJECTIVE 

119 

7.14 

0.0 

0.0 

2.50 

99.02 

8a 

3 

78.57 

71.4%3 

58.33 

70.00 

2.82 

8b 

9 

21.43 

14.29 

8.33 

15.00 

5.60 

8c 

10 

64.29 

35.71 

58.33 

52.50 

9.52 

TOTAL 

OBJECTIVE 

22 

35.71 

28.57 

16.67 

27.50 

17.95 

AVERAGE 

PERCENT 

ABOVE  CUT- 
OFF ACROSS 
ALL  TOTAL 
OBJECTIVES 

869 

7.14 

f 

5.10 

2.38 

5.00 

711.55 
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APPENDIX  E 

CORRELATIONS  BETWEEN  TOTAL  TEST  PERFORMANCE  AND  SELECTED 
BACKGROUND  VARIABLES  OF  REVISED  COURSE  PERSONNEL  (N=40) 


VARIABLE 

CORRELATION  VALUE 

LEVEL  OF  SIGNIFICANCE 

MEAN  SCORE 

VE 

0.37 

0.02 

92.8 

EL 

0.46 

0.01 

87.55 

CL 

0.46 

0.01 

99.35 

GT 

0.55 

0.01 

88.37 

ED 

0.23 

NS 

11.57 

0.48 

0.01 

28.97 

TOTAL 

PERFORMANCE 

— 

401.62 

